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Abstract 



^. With this paper, we contribute to the understanding of ant colony opti- 

• *H mization (ACO) algorithms by formally analyzing their runtime behavior. 

p\ We study simple MAX-MIN ant systems on the class of linear pseudo- 

Boolean functions defined on binary strings of length n. Our investiga- 
tions point out how the progress according to function values is stored in 
pheromone. We provide a general upper bound of 0{{n 3 log n)/ p) for two 
ACO variants on all linear functions, where p determines the pheromone 
update strength. Furthermore, we show improved bounds for two well- 
known linear pseudo-Boolean functions called OneMax and BinVal and 
give additional insights using an experimental study. 



1 Introduction 

Ant colony optimization (ACO) is an important class of stochastic search al- 
gorithms that has found many applications in combinatorial optimization as 
well as for stochastic and dynamic problems [3j. The basic idea behind ACO is 
that ants construct new solutions for a given problem by carrying out random 
walks on a so-called construction graph. These random walks are influenced 
by the pheromone values that are stored along the edges of the graph. During 
the optimization process the pheromone values are updated according to good 
solutions found during the optimization which should lead to better solutions 
in further steps of the algorithm. 

Building up a theoretical foundation of this kind of algorithms is a challeng- 
ing task as these algorithms highly rely on random decisions. The construction 
of new solutions highly depends on the current pheromone situation in the used 
system which highly varies during the optimization run. Capturing the theoret- 
ical properties of the pheromone constellation is a hard task but very important 
to gain new theoretical insights into the optimization process of ACO algorithms. 

With this paper, we contribute the theoretical understanding of ACO algo- 
rithms. Our goal is to gain new insights into the optimization process of these 
algorithms by studying them on the class of linear pseudo-Boolean functions. 
There are investigations of different depths on the behavior of simple evolu- 
tionary algorithms for this class of functions. The main result shows that each 
linear pseudo-Boolean function is optimization in expected time 0{n log n) by 
the well known (1+1) EA HEUST 

With respect to ACO algorithms, initial results on simplified versions of the 
MAX-MIN ant system [15] have been obtained. These studies deal with specific 
pseudo-Boolean functions defined on binary strings of length n. Such studies are 
primary focused on well-known linear example functions called OneMax and 
BinVal or the function LeadingOnes [T31 M 03 QH H2] • Recently, some results 
for ACO algorithms on combinatorial optimization problems such as minimum 
spanning trees [14] or the traveling salesman [10] have been obtained. These 
analyses assume that the pheromone bounds are attained in each iteration of 
the algorithms. This is the case if the pheromone update if a MAX-MIN ant 
system uses a strong pheromone update which forces the pheromone only to 
take on the maximum and minimum value. The analyses presented in |14j and 
|10j do not carry over to smaller pheromone updates. In particular, there are 
no corresponding polynomial upper bounds if the number of different function 
values is exponential with respect to the given input size. 

We provide new insights into the optimization of MAX-MIN ant systems for 
smaller pheromone updates on functions that may attain exponentially many 
functions values. Our study investigates simplified versions of the MAX-MIN 
ant system called MMAS* and MMAS t llJ on linear pseudo-Boolean func- 
tions with non-zero weights. For these algorithms, general upper bounds of 
0((n + (log n)/p)D) and 0(((n 2 log n)/p)D) respectively, have been provided 
for unimodal functions attaining D different function values |llj . As linear 
pseudo-Boolean function are unimodal, these bounds carry over to this class 
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Figure 1: Construction graph for pseudo-Boolean optimization with n = 5 bits. 

of functions. However, they only give weak bounds for linear pseudo-Boolean 
functions attaining many function values (e. g. for functions where the number 
of different function values is exponential in n) . 

We show an upper bound of 0((n 3 log n)/p) for MMAS* and MMAS opti- 
mizing each linear pseudo-Boolean function. Furthermore, our studies show that 
the method of fitness-based partitions may also be used according to pheromone 
value, i. e., MAX-MIN ant systems store the progress made according to function 
values quickly into a pheromone potential. This is one of the key observations 
that we use for our more detailed analyses on OneMax and BinVal in which 
we improve the results presented in |llj . 

To provide further insights that are not captured by our theoretical analy- 
ses, we carry out an experimental study. Our experimental investigations give 
comparisons to simple evolutionary algorithms, and consider the impact of the 
chosen weights of the linear functions and pheromone update strength with re- 
spect to the optimization time. One key observation of these studies is that 
OneMax is not the simplest linear function for the simple MAX-MIN ant sys- 
tems under investigation. Additionally, the studies indicate that the runtime 
grows at most linearly with 1/p for a fixed value of n. 

We proceed as follows. In Section [2] we introduce the simplified MAX-MIN 
ant systems that are subject to our investigations. In Section [3j we provide 
general runtime bounds for the class of linear pseudo-Boolean functions and 
present specific results for OneMax and BinVal in Sectionffl Our experimental 
study which provides further insights is reported in Section[5] Finally, we discuss 
our results and finish with some concluding remarks. 

2 Simplified MAX-MIN Ant Systems 

We first describe the simplified MAX-MIN ant systems that will be investigated 
in the sequel. The following construction graph is used to construct solutions 
for pseudo-Boolean optimization, i.e., bit strings of n bits. It is based on a 
directed multigraph C = (V,E). In addition to a start node vq, there is a node 
Vi for every bit i, 1 < i < n. This node can be reached from Vi-i by two edges. 
The edge ej,! corresponds to setting bit i to 1, while e,,o corresponds to setting 
bit i to 0. The former edge is also called a 1-edge, the latter is called 0-edge. 
An example of a construction graph for n = 5 is shown in Figure [T] 



In a solution construction process an artificial ant sequentially traverses 
the nodes vq, vi, . . . , v n . The decision which edge to take is made according 
to pheromones on the edges. Formally, we denote pheromones by a func- 
tion r: E — >• RJ. From v^i the edge en is then taken with probability 
r ( e i,i)/( r ( e i,o) + r ( e i,i))- I n the case of our construction graph, we identify 
the path taken by the ant with a corresponding binary solution x as described 
above and denote the path by P(x). 

All ACO algorithms considered here start with an equal amount of 
pheromone on all edges: T(e^o) = r (<2i,i) = 1/2. Moreover, we ensure that 
T ( e i,o) + T ( e i,i) — 1 holds, i.e., pheromones for one bit always sum up to 1. 
This implies that the probability of taking a specific edge equals its pheromone 
value; in other words, pheromones and traversion probabilities coincide. 

Given a solution x and a path P(x) of edges that have been chosen in the 
creation of x, a pheromone update with respect to x is performed as follows. 
First, a p- fraction of all pheromones evaporates and a (1 — /^-fraction remains. 
Next, some pheromone is added to edges that are part of the path P(x) of x. 
To prevent pheromones from dropping to arbitrarily small values, we follow the 
MAX-MIN ant system by Stutzle and Hoos [H] and restrict all pheromones to 
a bounded interval. The precise interval is chosen as [1/n, 1 — 1/n]. This choice 
is inspired by standard mutations in evolutionary computation where for every 
bit an evolutionary algorithm has a probability of 1/n of reverting a wrong 
decision. 

Depending on whether an edge e is contained in the path P(x) of the solution 
x, the pheromone values r are updated to t' as follows: 

r'(e) = min < (1 — p) ■ r(e) + p, 1 > if e £ P(x) and 

I n) 

r'(e) =max { (1 - p) ■ r(e), - 1 if e 4. P(x). 

I n ) 

The algorithm MMAS now works as follows. It records the best solution 
found so far, known as best-so-far solution. It repeatedly constructs a new 
solution. This solution is then compared against the current best-so-far and 
it replaces the previous best-so-far if the objective value of the new solution is 
not worse. Finally, the pheromones are updated with respect to the best-so-far 
solution. A formal description is given in Algorithm [T] 

Note that when a worse solution is constructed then the old best-so-far 
solution is reinforced again. In case no improvement is found for some time, 
this means that the same solution x* is reinforced over and over again and the 
pheromones move towards the respective borders in x* . Previous studies [6] 
have shown that after (\nn)/p iterations of reinforcing the same solution all 
pheromones have reached their respective borders. This time is often called 
"freezing time" |11) . 

We also consider a variant of MMAS known as MMAS* [11] (see Algo- 
rithm!^. The only difference is that the best-so- far solution is only changed in 
case the new solution is strictly better. In application of ACO often this kind 



Algorithm 1 MMAS 
1: Set T( u , u ) = 1/2 for all (u,v) £ E. 



Construct a solution x" . 
Update pheromones w.r.t. x* . 
repeat forever 

Construct a solution x. 

if /(#) > f(x*) then x* := £. 

Update pheromones w. r. t. x* 



of strategy is used. However, in [TT] it was argued that MMAS works better on 
functions with plateaus as MMAS is able to perform a random walk on equally 
fit solutions. 



Algorithm 2 MMAS* 



Set T(u, v ) — 1/2 for all (u, v) £ E. 
Construct a solution x* . 
Update pheromones w. r. t. x* . 
repeat forever 

Construct a solution x. 

if f(x) > f(x*) then x* := x. 

Update pheromones w. r. t. x* 



In the following we analyze the performance of MMAS and MMAS* on linear 
functions. We are interested in the number of iterations until the first global 
optimum is found. This time is commonly called optimization time. 

Note that the pheromones on the 1-edges e. i suffice to describe all 
pheromones as r(e^o)+' r (ej,i) = 1. When speaking of pheromones, we therefore 
often focus on pheromones on 1-edges. 

3 General Results 

We first derive general upper bounds on the expected optimization time of 
MMAS and MMAS* on linear pseudo-Boolean functions. A linear pseudo- 
Boolean function for an input vector x — (x\, . . . , x n ) is a function / : {0, 1} i— > 
K, with f(x) = Ylj—i WiXi and weights Wi £ M. 

We only consider positive weights since a function with a negative weight 
Wi may be transformed into a function with a positive weight w^ = — Wi by 
exchanging the meaning of bit values and 1 for bit i. This results in a function 
whose value is by an additive term of w' t larger. This and exchanging bit values 
does not impact the behavior of our algorithms. We also exclude weights of 
as these bits do not contribute to the fitness. Finally, in this section we assume 
without loss of generality that the weights are ordered according to their values: 
W\ > W2 > • ■ • > w n . 



Two well-known linear functions are the function OneMax where w\ = 
w-2 = ■ ■ ■ = w n = 1 and the function BinVal where Wi = 2™ _I . These functions 
represent two extremes: for OneMax all bits are of equal importance, while in 
BinVal a bit at position i can dominate all bits at positions i + 1, . . . , n. 

3.1 Analysis Using Fitness-based Partitions 

We exploit a similarity between MMAS, MMAS* and evolutionary algorithms to 
obtain a first upper bound. We use the method of fitness-based partitions, also 
called fitness-level method, to estimate the expected optimization time. This 
method has originally been introduced for the analysis of elitist evolutionary 
algorithms (see, e. g., Wegener [16]) where the fitness of the current search point 
can never decrease. The idea is to partition the search space into sets A\,..., A m 
that are ordered with respect to fitness. Formally, we require that for all 1 < 
i < m — 1 all search points in At have a strictly lower fitness than all search 
points in Aj+i. In addition, A m must contain all global optima. 

Now, if Si is (a lower bound on) the probability of discovering a new search 
point in A; +1 U- • -UA m , given that the current best solution is in A il the expected 
optimization time is bounded by y^._ 1 1/sj as 1/s, is (an upper bound on) the 
expected time until fitness level i is left and each fitness level has to be left at 
most once. 

Gutjahr and Sebastiani [6] as well as Neumann, Sudholt, and Witt [Hj have 
adapted this method for MMAS*. If the algorithm does not find a better search 
point for some time, the same solution x* is reinforced over and over again, 
until eventually all pheromones attain their borders corresponding to the bit 
values in x*. We say that then all pheromones are saturated. In this setting 
the solution creation process of MMAS* equals a standard bit mutation of x* in 
an evolutionary algorithm. If s, is (a lower bound on) the probability that the 
a mutation of x* creates a search point in A i+ i U • • • U A m , then the expected 
time until MMAS* leaves fitness level Ai is bounded by (lnn)/p + 1/sj as either 
the algorithm manages to find an improvement before the pheromones saturate 
or the pheromones saturate and the probability of finding an improvement is at 
least Si. This results in an upper bound of m- (hin)/p+^™T L l/s{ for MMAS*. 

One restriction of this method is, however, that fitness levels are only allowed 
to contain a single fitness value; in the above bound m must equal the number 
of different fitness values. Without this condition — when a fitness level contains 
multiple fitness values — MMAS* may repeatedly exchange the current best- 
so-far solution within a fitness level. This can prevent the pheromones from 
saturating, so that the above argument breaks down. For this reason, all upper 
bounds in llj grow at least linearly in the number of function values. 

The following lemma gives an explanation for the time bound (Inn) //a for 
saturating pheromones. This time is also called freezing time. We present a 
formulation that holds for arbitrary sets of bits. Though we do not make use of 
the larger generality, this lemma may be of independent interest. 

Lemma 1. Given an index set I C {1, . . . , n} we say that a bit is in I if its 



index is in I . Let x* be the current best-so-far solution of MMAS or MMAS* 
optimizing an arbitrary function. After (lnn)/p further iterations either all 
pheromones corresponding to bits in I have reached their respective bounds in 
{1/n, 1 — 1/n} or x* has been replaced by some search point x** with f(x**) > 
f(x*) for MMAS and f{x**) > f(x*) for MMAS* such that x** differs from x* 
in at least one bit in I. 

Proof. Assume the bit values of the bits in / remain fixed in the current best- 
so-far solution for (lnn)/p iterations as otherwise there is nothing to prove. In 
this case for every bit x* with i G / the same bit value x* has been reinforced 
for (lnn)/p iterations. This implies that for the edge in the construction graph 
representing the opposite bit value the lower pheromone border 1/n has been 
reached, as for any initial pheromone < r^ < 1 on this edge we have (1 — 
p)( lnn VPTi< e- lnn n< 1/n. □ 

So far, the best known general upper bounds for MMAS and MMAS* that 
apply to every linear function are 0((n 2 log n)/p ■ 2") and 0((lagn)/p ■ 2"), 
respectively, following from upper bounds for unimodal functions |llj . The term 
2™ results from the fact that in the worst case a linear function has 2™ different 
function values. This is the case, for instance, for the function BinVal. An 
exponential upper bound for linear functions is, of course, unsatisfactory. The 
following theorem establishes a polynomial upper bound (with respect to n and 
1/p) for both algorithms. 

Theorem 2. The expected optimization time of MMAS and MMAS* on every 
linear function is in 0((n 3 log n)/p). 

Proof. The proof is an application of the above-described fitness-based parti- 
tions method. In the first step, we consider the time needed to sample a solution 
which is at least on the next higher fitness level. We analyze the two situations 
when the pheromones are either saturated or not. Our upper bound is the result 
of the repetition of such advancing steps between fitness levels. 
We define the fitness levels 



A i = \xG {0, 1}" 



i+i 



3=1 3 = 1 



and apply the fitness-based partitions method. Intuitively, the fitness levels are 
defined such that a solution x is at least on fitness level Ai if the leftmost bit of 
x with value is at position i + 1. 

The expected time spent sampling solutions on fitness level Ai (i. e., without 
sampling a solution of a higher fitness level) is the combination of the time 
spent in Ai with saturated pheromone values and the time spent in Ai with 
unsaturated pheromone values. In the following, we analyze for both situations 
the probabilities to sample a solution of a higher fitness level. In the end, as 
MMAS and MMAS* might not remain in one of both situations exclusively, but 



alternates between situations of saturated and unsaturated pheromone values, 
we take the sum of both runtimes as an upper bound. 

First, when the pheromone values are saturated, the probability to flip the 
leftmost zero and keep all other bits is 1/n • (1 — l/n)™ -1 > 1/n ■ 1/e, as 
(1 - 1/n)™" 1 > 1/e > (1 - 1/n)" holds for all n € M. This results in a 
probability of Vt(l/n) of advancing in such a situation. Thus, the algorithm stays 
an expected number of O(n) steps with saturated pheromone values without 
sampling a solution on a higher fitness level. 

For the second case, when the pheromone values are not saturated, let i < n, 
and suppose x* £ A4 is the current best solution. Then, let us denote by 
G = U,>i A; a ^ good solutions that are at least on the next higher fitness 
level, and by B = {x £ {0, 1}" \f(x) > f(x*),x </ G} all bad solutions that are 
in Ai with an equal or a higher function value than x* . Thus, every improving 
sampled solution belongs to G U B. 

Let h : {0, 1}" >->• {0, 1}™ be the function that returns for a given solution x 
a solution x' £ G, where the leftmost was flipped to 1. 

Let P(x) be the probability to sample a new solution x. Then the probability 
q to sample any of a; and h(x) is greater than or equal to P{x). The probability to 
sample h{x) is the probability to sample any of x and h(x) times the probability 
that the leftmost zero of x was sampled as a one. Thus, for all x, P(h{x)) = 
q ■ 1/n > P(x)/n, as the pheromone values are at least 1/n. 

Furthermore, each solution h(x) has at most n preimages with respect to h. 
Note that, for all x e B, h(x) e G. 

Thus, the probability to sample the next solution x G G is 



P(x £G) = J2 P ( x ) ^ 



E x e B P(Hx)) 



> 



xeG 

E x e B P(x) _P(x£B) 



So, sampling a good solution is at least 1/n 2 times as likely as sampling a 
bad one. 

Furthermore, while no bad solutions are sampled, at most (lnn)/p steps are 
spent before the pheromone values are saturated (based on Lemma nj. 

Thus, up to (Inn)//) steps are spent with unsaturated pheromone values 
before sampling a new solution, and 0((n 2 log ri)/p) steps are spent in total 
sampling solutions in B before sampling a solution in G. 

Consequently, the time spent on one fitness level is the sum of the times spent 
in either situation of the pheromone values, that is, 0{n) + 0{(n 2 log n)/p) = 
0((n 2 log n)/p). 

Finally, as there are n fitness levels, the afore-described steps have to be 
performed at most n-times, which yields a total runtime of 0((n 3 logn)/p). □ 



3.2 Fitness-based Partitions for Pheromones 

We describe an approach for extending the argument on /-based partitions 
to pheromones instead of the best-so-far solution. This alternate approach is 
based on weighted pheromone sums. Given a vector of pheromones r and a 
linear function /, the weighted pheromone sum (wps) is /(r). 

The idea is that, during a run of the algorithm, the wps should rise until 
saturated with respect to the current best search point, and then a significant 
improvement should have a decent probability. 

Define a function v on bit strings as follows. 

~i -Wi, otherwise. 

A pheromone vector r is called saturated with respect to a search point x* 
iff 

f(r)>v(x*). 

Note that this definition of saturation is very much different from previous 
notions of saturation in the literature. 

Let, for all i, a^ be the bit string starting with i Is and then having only Os. 
We let 

A % = {x | /(ai) < f{x) < f(a i+1 )} 

and 

B, = {t I v(a t ) < /(t) < v(a l+1 )}. 

While [Ai)i captures the progress of the search points towards the optimum, 
(-Bi)i captures the progress of the pheromones. 

Lemma 3. For all i, if the best-so-far solution was in [J^^Aj for at least 
(lnn)//9 iterations, then t € -Bj. 

Proof. Let h be such that Vs : h(s) = s(l — p) + p. Let t° be the vector of 
pheromones when the algorithm samples a solution in lj,>j Aj for the first time, 
and let (r t ) t be the vectors of pheromones in the successive rounds. Further, 
we define the sequence of capped-out pheromones (r cap, ') f such that, for all i, 
cap, __ jj-^jj^^/j* (1/rz) , Tj). For this capped-out version of pheromones we have, 
for all t with h t+1 (l/n) < 1 - 1/n, 

/(r cap '* +1 )>(l-/t>)/(r cap '*)+/t./(o i ) ) 

as pheromones will evaporate and at least an /(a^) weighted part of them will 
receive new pheromone p (note that the old capped-out pheromone raised by h 
cannot exceed the new cap). Thus, we get inductively 

Vt : /(r ca P<<) > fc*(l/n)/(o«). 

As we know from Lemma n] for t > (Inn)//? we have /i'(l/n) > 1 — 1/n, and, 
thus, t* e B,. □ 



This argument opens up new possibilities for analyses and we believe it to 
be of independent interest. 

If it is possible to show, for all i, if r 6 Bi, then the probability of sampling 
a new solution in U 7>i Aj is f2(l/n), then LemmapWould immediately improve 
the bound in Theorem 2 to O (n 2 + (nlogn)/ p) . However, this claim is not 
easy to show. 

It is possible to prove this for the special case of OneMax using the following 
theorem by Gleser [5]. This theorem gives a very nice handle on estimating 
probabilities for sampling above-average solutions for OneMax. 

Theorem 4 (Gleser [5 ). Let t,t' be two pheromone vectors such that, for all 
j < n, the sum of the j least values of r is at least the sum of the j least values 
of t' . Let X be the sum of the elements of t' . Then it is at least as likely to 
sample [A + lj Is with r as it is with r' . 

We can use this theorem to get a good bound for OneMax: the worst case 
for the probability of an improvement is attained when all but at most one 
pheromones are at their respective borders. In this situation, when there are 
still i Is missing, the probability of an improvement is Cl(i/ri). Combining this 
with Lemma [3j we get the following result. 

Corollary 5. The expected optimization time of MM AS and MMAS* on One- 
Max is bounded by 



»E 



n \ 

— h n ■ (Inn) / p — 0((nlogn)/p). 

i I 



This re-proves the 0((rtlogn)/p)-bound for MMAS* in [TT] and it improves 
the current best known bound for MMAS on OneMax by a factor of n 2 . We 
will present an even improved bound for OneMax in Section [4T] 

However, with regard to general linear functions it is not clear how this 
argument can be generalized to arbitrary weights. In the following section we 
therefore turn to the investigation of concrete linear functions. 

4 Improved Bounds for Selected Linear Func- 
tions 

Using insights from Section pT2| w e next present improved upper bounds for the 
function OneMax (Section |4.1[ ). Afterwards, we focus on the special function 
BinVal (Section |4.2[ ). These resulting bounds are much stronger than the 
general ones given in Section [3] above. 

4.1 OneMax 

Recall the bound 0((nhgn)/p) for MMAS and MMAS* from Corollary [5] In 
the following we prove a bound of 0{n log n + n/p) for both MMAS and MMAS* 



10 



by more detailed investigations on the pheromones and their dynamic growth 
over time. This shows in particular that the term 1/p has at most a linear 
impact on the total expected optimization time. 

Let v and a^ be as in Section 3.2 For all i < n, we let «j = i(l — 1/n). 
Observe that v(a,i) = ct: h + (n — i)/n and, in particular, c^ < v(a.i) < a t + 1. 

The following lemma gives a lower bound on the pheromone f(r + ) after on 
iteration. Note that for OneMax /(t + ) corresponds to the sum of pheromones. 

Lemma 6. Let i < j and let t be the current pheromones with v(a,i) < / (t) < 
u(aj-|-i) and suppose that the best-so-far solution has at least j ones. We denote 
by t + the pheromones after one iteration of MMAS or MMAS*. Then we have 
/(r+) > v(a i+1 ) or 

f(r+) - a t > (/(r) - a<)(l - p) + (j - i)p > w(a<). (1) 

Proof. Suppose that in rewarding bit positions from r to get r + , exactly k 
positions cap out at the upper pheromone border 1 — 1/n. From f(r + ) < v(a,i+i) 
(otherwise there is nothing left to show), we have k < i. We decompose f(r + ) 
into the contribution of the capped-out bits (which is «&, being k of the j 
rewarded positions) and the rest. Then we have 

/(r+) > a k + (/(r) - a fc )(l - p) + (j - k)p. 

We now get 

"j + (/(t) - c*i)(l - p) + (j - i)p 
= a k + cti- k + (f(r) -ak- a»-jfe)(l - p) + (j - fc)p + (k - i)p 

< /( r+ ) + «»-fc - "i-fel 1 - p) + ( k - i)P 
= /(r + )+p(oi_ h -(i-fc)) 

< /(t + ). 

From u(ai) < /(r) we get 

/(r+) > a, + (/(t) - Oi)(l - p) + (j - *)P 

> aj + (w(aj) - ai)(l - p) + (j - i)p 
= v(a,i) + p{j - i - v{ai) + cti). 

From vifli) — cti < 1 and j > i we get the desired conclusion. □ 

One important conclusion from Lemma[6]is that once the sum of pheromones 
is above some value v(a,i), it can never decrease below this term. 

Now we extend Lemma [6] towards multiple iterations. The following lemma 
shows that, unless a value of v(aj+i) is reached, the sum of pheromones quickly 
converges to <x, when j is the number of ones in the best-so-far solution. 

Lemma 7. Let i < j and let t be the current pheromones with v(a,i) < /(t) < 
v(a,i+i) and suppose that the best-so-far solution has at least j ones. For all t, 
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we denote by r' the pheromones after t iterations of MMAS or MMAS*. Then 
we have for all t /(r*) > v(ai+i) or 

f(r t )-a i >(j-i)(l-Q.-p) t ). 

Proof. Inductively for all t, we get from Lemma^/fr') > v(a i+ i) or 

t-i 
fir 1 ) -a % > (/(r°) - a,)(l pf + (j i)p£(l - pf 

= (/(r°) - ai )(l pf + (j ») / 1 ~ ( ( 1 1 ~^ 
= (/(r°)-o l )(l-p) t + (j- J )(l-(l-p) t ) 

>(j-i)(i-(i -,>)'). 

D 

Theorem 8. TTie expected optimization time of MMAS and MMAS* on One- 
Max is 0(n log n + n/p). 

Proof. Define v(a_i) = 0. Let r be the current pheromones and r be the 
pheromones after £ iterations of MMAS or MMAS*. We divide a run of MMAS 
or MMAS* into phases: the algorithm is in Phase j if /(r) > /(a_j_i), the 
current best-so-far solution contains at least j ones, and the conditions for 
Phase j + 1 are not yet fulfilled. We estimate the expected time until each 
phase is completed, resulting in an upper bound on the expected optimization 
time. 

We first deal with the last n/2 phases and consider some Phase j with 
j > n/2. By Lemma [Rafter t iterations we either have /(r*) > v(aj + i) or 
/(t*) > aj-i + 1 - (1 - pf. Setting t := \l/p\, this implies /(r*) > a,_i + 
1 — e _pt > olj-\ + 1 — 1/e. We claim that then the probability of creating j + 1 
ones is r2((n — j)/n). 

Using j > n/2, the total pheromonc <x,-i + 1 — 1/e can be distributed on an 
artificially constructed pheromone vector r' as follows. We assign value 1 — 1/n 
to j — 1 entries and value 1/n to n — j entries. As (n — j)/n < 1/2, we have 
used pheromone of ctj-i + 1/2 and so pheromone 1 — 1/2 — 1/e = £1(1) remains 
for the last entry. We now use Theorem [4] to see that it as least as likely to 
sample a solution with j + 1 ones with the real pheromone vector r* as it is with 
t'. By construction of t' this probability is at least (1 — l/ny~ l ■ (1/2 — 1/e) • 
(n — j)/n ■ (1 — l/n) n ^' J ^ 1 > (n — j)(l/2 — l/e)/(en) as a sufficient condition is 
setting all bits with pheromone larger than 1/n in r' to 1 and adding exactly 
one 1-bit out of the remaining n — j bits. 

Invoking Lemma [7] again for at least j + 1 ones in the best-so- far solution, 
we get f(T t+t ') > aj-i + 2(1 -(1-p)*'), which fori' := \2/p\ yields /(r*+*') > 
<X/_i + 3/2 > ctj + 1/2 > v(aj) as j > n/2. 

For the phases with index j < n/2 we construct a pessimistic pheromone 
vector t' in a similar fashion. We assign value 1 — 1/n to j — 2 entries, value 
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1/n to n — j entries, and put the remaining pheromone on the two last bits 
such that either only one bit receives pheromone above 1/n or one bit receives 
pheromone 1 — 1/n and the other bit gets the rest. To show that the pheromones 
raise appropriately, we aim at a larger gain in the best number of ones. The 
probability of constructing at least j + 2 ones with any of the above-described 
vectors is at least (1 - l/n) J '~ 2 • ( n ^' +2 ) ■ 1/n 3 • (1 - l/n) n_ J > l/(48e) = 
0((n-j)/n). 

Using the same choice t' := [2/p] as above, Lemma M yields /(t* + * ) > 
(Xj-i + 3(1 - (1 - pY) > aj-i + 2 > v{a 3 ). 

Summing up the expected times for all phases yields a bound of 




4.2 BinVal 

The function BinVal has similar properties as the well-known function 
LeadingOnes(:e) := Ym=i IIi^i x j that counts the number of leading ones. 
For both MMAS and MMAS* the leading ones in x* can never be lost as set- 
ting one of these bits to will definitely result in a worse solution. This implies 
for both algorithms that the pheromones on the first LeadingOnes(x*) bits 
will strictly increase over time, until the upper pheromone border is reached. 
In [TTj the following upper bound for LeadingOnes was shown. 



Theorem 9 (Q3]). The expected optimization time of MMAS and MMAS* 
on LeadingOnes is bounded by 0(n 2 + n/p) and o(n 2 ■ {l/p) e + lo "/ / \ ) for 
every constant e > 0. 

The basic proof idea is that after an average waiting time of £ iterations 
the probability of rediscovering the leading ones in x* is at least Q(e~ 5 ^ ep " > ). 
Plugging in appropriate values for £ then gives the claimed bounds. 

Definition 10. For £ G IN and a sequence of bits x\ 1 . . . , Xi ordered with respect 
to increasing pheromones we say that these bits form an (i,£)-layer if for all 

Tj > min(l - 1/n, 1 - (1 - P y e ). 

With such a layering the considered bits can be rediscovered easily, depend- 
ing on the value of t. The following lemma was implicitly shown in [111 proof 
of Theorem 6] . 

Lemma 11. The probability that in an (i,£)-layer all i bits defining the layer 
are set to 1 in an ant solution is Sl(e~ 5 '^ p '). 
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Assume we have k = LeadingOnes(:e*) and the pheromones form a (k,£)- 



layer. Using Lemma 11 and the fact that a new leading one is added with proba- 
bility at least 1/n, the expected waiting time until we have LeadingOnes(:e*) > 
fc + 1 and a (k + 1, ^)-layer of pheromones is at most 0(n ■ e 5 /^ p ' + £). As this 
is necessary at most n times, this gives us an upper bound on the expected 
optimization time. Plugging in £ = \5/p] and £ = \5/(ep\n(l/p))~\ yields the 
same upper bounds for BinVal as we had in Theorem [9] for LeadingOnes. 

Theorem 12. The expected optimization time of MMAS and MMAS* on Bin- 
Val is bounded by 0(n 2 + n/ p) and Oin 2 - (1/ p) e + t "/ / -, ) for every constant 
e > 0. 

These two bounds show that the second term "+n/p" in the first bound — 
that also appeared in the upper bound for OneMax — can be lowered, at the 
expense of an increase in the first term. It is an interesting open question 
whether for all linear functions when p is very small the runtime is o(n/ p) 1 i. c., 
sublinear in 1/p for fixed n. The relation between the runtime and p is further 
discussed from an experimental perspective in the next section. 

5 Experiments 

In this section, we investigate the behavior of our algorithms using experimental 
studies. Our goal is to examine the effect of the pheromone update strength as 
well as the impact of the weights of the linear function that should be optimized. 
Related to our experiments are those in [TJ [IT] . The authors of the first 
article concentrate their analyses of 1-ANT and MMAS on OneMax, Leading- 
Ones, and random linear functions. A single n for each function is used, and a 
number of theory-guided indicators monitors the algorithms' progress at every 
time step, in order to measure the algorithms' progress within individual runs. 
In the second article, the runtime of MMAS and MMAS* is investigated on 
OneMax and other functions, for two values of n and a wide range of values of 

P- 

First, we investigate the runtime behavior of our algorithms for different 
settings of p (see Figure [2]). We investigate OneMax, BinVal, and random 
functions where the weights are chosen uniformly at random from the interval 
]0, 1] . For our investigations, we consider problems of size n = 200, 250, . . . , 1000. 
Each fixed value of n is run for p E {1.0,0.5,0.1,0.05} and the results are 
averaged over 1000 runs. Remember that, for p = 1.0, MMAS* is equivalent 
to the (1+1) EA*, and MMAS is equivalent to the (1+1) EA (see [5] for the 
definition of the evolutionary algorithms) . 

One general observation is that the performance of MMAS* on random lin- 
ear functions and BinVal is practically identical to that of MMAS. This was 
expected, as several bit positions have to have the same associated weights in 
order for MMAS to benefit from its weakened acceptance condition. Further- 
more, we notice that OneMax is not the simplest linear function for MMAS* 
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Figure 2: Runtime of MMAS* (left column) and MMAS (right column). 
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to optimize. In fact, for certain values of p, OneMax is as difficult to optimize 
as BinVal. 

For all experiments, the performance of MMAS* with p = 1.0 is very close to 
that of MMAS with p = 1.0. However, with different values of p, several perfor- 
mance differences are observed. For example, MMAS* with p = 0.5 and p = 0.1 
optimizes random linear functions faster than MMAS* with p — 1.0, which is on 
the other hand the fastest setup for OneMax. Furthermore, the performance of 
MMAS increased significantly with values of p < 1.0, e. g., MMAS with p = 0.1 
is 30% faster than MMAS with p = 1.0. Another general observation is that 
MMAS* performs better on random linear functions, than on OneMax, e. g. 
for n = 1000 and p = 0.1 the runtime decreases by roughly 10%. 

In the following, we give an explanation for this behavior. During the opti- 
mization process, it is possible to replace a lightweight 1 at bit i (i. e., a bit with 
a relatively small associated weight wi) with a heavyweight at bit j (i. e., a bit 
with a relatively large associated weight «;,•). Afterwards, during the freezing 
process, the probability for sampling again the lightweight 1 at bit i (whose as- 
sociated Ti is in the process of being reduced to T m - nl ) is relatively high. Unlike in 
the case of OneMax, it is indeed possible for MMAS* to collect heavyweight Is 
in between, and the "knowledge" of the lightweight Is is available for a certain 
time, stored as a linear combination of the pheromones' values. This effect oc- 
curs in the phase of adjusting the pheromones, not when the pheromone values 
are saturated. Otherwise, the effect could be observed for the (1+1) EA and 
the (1+1) EA* as well. 

We have already seen that the choice of p may have a high impact on the 
optimization time. The runtime bounds given in this paper increase with de- 
creasing p. In the following, we want to investigate the impact of p closer by 
conducting experimental studies. We study the effect of p by fixing n = 100 and 
varying p = 1/x with x = 1, 11, . . . , 1001. The results are shown in Figurep^and 
are averaged over 10.000 runs. The effect that small values of p can improve 
the performance on OneMax is observable again. The fitted linear regression 
lines, which are based on the mean iterations for 1/p € (500, 1000], support our 
claim that the runtime grows at most linear with 1/p for a fixed value of n. 
In fact, the fitted lines indicate that the growth of the average runtime is very 
close to a linear function in 1/p. The real curves appear to be slightly concave, 
which corresponds to a sublinear growth. However, the observable effects are 
too small to allow for general conclusions. 

6 Conclusions and Future Work 

The rigorous analysis of ACO algorithms is a challenging task as these algo- 
rithms are of a high stochastic nature. Understanding the pheromone update 
process and the information that is stored in pheromone during the optimization 
run plays a key role to increase their theoretical foundations. 

We have presented improved upper bounds for the performance of ACO 
on the class of linear pseudo-Boolean functions. The general upper bound of 
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Figure 3: Impact of pheromone evaporation factor in MMAS* (left column) and 
MMAS (right column). 
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0((n 3 log n)/p) from Theorem ^applies to all linear functions, but in the light of 
the smaller upper bounds for OneMax and BinVal we believe that this bound 
is still far from optimal. Stronger arguments are needed in order to arrive at a 
stronger result. 

We also have developed novel methods for analyzing ACO algorithms with- 
out relying on pheromones freezing at pheromone borders. Fitness-level argu- 
ments on a pheromone level have revealed one possible way of reasoning. For 
OneMax this approach, in combination with results from [5], has led to a 
bound of 0(nlogn + n/p), both for MMAS and MMAS*. This is a major im- 
provement to the previous best known bounds 0((n 3 log n)/p) for MMAS and 
O ((n log n)/p) for MMAS* and it finally closes the gap of size n 2 between the 
upper bounds for MMAS and MMAS*. We conjecture that our improved bound 
holds for all linear functions, but this is still a challenging open problem. 

The experimental results have revealed that a slow adaption of pheromone 
is beneficial for MMAS on OneMax as MMAS was faster than the (1+1) EA 
for all investigated p-values not larger than 0.5. We also argued why MMAS* is 
faster on random-weight linear functions than on OneMax. The experiments 
also gave more detailed insights into the impact of the evaporation factor on the 
average runtime. 

We conclude with open questions and tasks for future work: 

1. Do MMAS and MMAS* optimize all linear pseudo-Boolean functions in 
expected time 0{nlogn + n/p)? 

2. Analyze ACO for combinatorial problems like minimum spanning trees 
and the TSP in settings with slow pheromone adaptation. 
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